In some MMS applications, the intention is to capture especially pictures of building facades and other fixed objects, like trees, street signs and street lamps, that are later used in “real-world” 2D and/or 3D images of streets used in e.g. car navigation systems. Then, these images are shown to drivers of a car provided with such a navigation system such that the driver sees 2D and/or 3D images on a screen of the navigation system corresponding with the real world view when looking through the windows of the car. Such pictures may also be used in other applications than car navigation systems, for instance, in games that can be played on computers either as a stand alone systems or as cooperating in a networked environment. Such an environment may be the Internet. The solution of the present invention as presented below is not restricted to a specific application.
However, millions of such MMS images may contain privacy information such as faces of people and readable license plates of cars that are unintentionally present on the images. It is desirable not to use such images in public applications with such privacy or other undesired information still intact. For instance, newspapers have reported about such undesired information being present in images used in street map views distributed by Google™. Images taken in a real world condition represent static and moving objects in the vicinity of the MMS. In the images, the objects having such privacy or other undesired information may be static or moving relative to the fixed world. One has to identify such objects in the images taken by the camera on the MMS. Some prior art applications have tried to identify moving objects only on the basis of image properties and determine their trajectory of movement based on color pixel properties in sequences of images. However, such an approach works only as long as objects can be determined on more then two images in sequence to determine the trajectory.
Others have disclosed systems in which other types of sensors are used to determine short time trajectory approximation of objects relative to a vehicle arranged with such sensors. Such sensors may include laser scanners, radar systems and stereo-video cameras. Such systems are, for instance, referred to in the introduction of EP 1 418 444. This document relates to real-time applications where relative positions and speeds of objects to the vehicle are important, for instance, in order to avoid accidents between the vehicle and the objects. The document does not disclose how the position and speed data obtained by the sensors can be mapped on image data obtained by the stereo-video cameras. Moreover, it does not disclose how to determine absolute positions and absolute speeds of such objects. Here, “absolute” is to be understood in the sense of being absolute relative to the fixed real world as determined by the earth and objects fixed to the earth, like buildings, traffic signs, trees, mountains, etc. Such a real world can be, for instance, defined by a reference grid as used by the GPS system. Moreover, this document does not address how to deal with privacy sensitive data in images taken by the video cameras.
Using laser scanner data to assist in identifying locations of building footprints is for example described in co-pending patent application PCT/NL2006/050264.